Multispectral (MS) imaging is a technique for capturing images of a scene and/or object at different and specific spectral wavelength ranges. The human eye is capable of observing the visible wavelength ranges of the electromagnetic (EM) spectrum, which is a very small portion of the entire EM spectrum. If a broadband, “white” light source, such as the Sun, is illuminating an object, that object would remit light wavelengths in addition to the visible wavelengths.
Measuring the certain characteristics of the light remitted by an object can provide clues about the objects intrinsic properties. For example, these properties can include the physical state or the molecular composition of the object observed, along with many other derived properties. Polarimetric imaging is the method of capturing images of a scene and/or object at different and specific light polarizations. Light travels as a transverse electromagnetic wave and can vibrate in different directions orthogonal to its direction of travel. These directions of vibration, or polarizations of light, can change as it passes through different materials due to their scattering characteristics and molecular structure. Light polarization analysis is used for identifying degree of depolarization heterogeneity amongst supposedly homogeneous scenes, objects and/or materials. The use of multispectral and polarimetric imaging has become common practice in many fields such as science, defense, security, food inspection, quality control, criminology, remote sensing, and medicine. The number of applications for such imaging systems is continuously growing.
There are several methods enabling MS imaging. The simplest method is to dissociate a color image, captured using a polychromatic camera, into its red, green, and blue (RGB) channels [1]. This method sub-divides the visible spectrum into three independent spectral bands. The spectral bands are highly dependent on the spectral response of the polychromatic detector used for imaging and will vary between different detectors by different manufacturers. This method is not a very accurate radiometric representation of field-of-view (FOV) because polychromatic detectors typically use a Bayer filter [2] to acquire the three channel RGB information and interpolate the missing spectral information in a given detector pixel using its neighboring pixels which do not contain similar spectral information.
There exist two alternate, and perhaps more accurate, methods for MS imaging. The first method employs a series of spectral bandpass filters combined with a monochromatic camera. These filters are designed to accurately transmit a wavelength range of interest while suppressing all other wavelengths. The filters can be placed in the path of the light entering the camera using a motorized filter wheel [3], or liquid-crystal tunable filters [4], or acousto-optical tunable filters [5]. The second method uses a series of light sources that can illuminate the target with light of a specific wavelength range. In this case the remitted light is acquired on a monochromatic camera [6, 7]. Both of these alternate methods require sequential imaging of the FOV which can be time consuming and non-preferential when imaging dynamic FOVs.
Simultaneous imaging is possible using various beamsplitter arrangements while imaging each spectral region on its own respective camera system [8]. A handheld MS imager for terrestrial applications available on the market is ADC by TETRACAM Inc. [9]. This device is capable of imaging three spectral bands spanning the visible to the near-Infrared (NIR) spectral range. There are reports of other hand-held multispectral imaging devices [10, 11, 12], but these systems only operate either in the visible or in the NIR spectral range and are not capable of stereoscopic imaging. Manakov developed a handheld MS imager using an augmentation to digital single lens reflex (DSLR) cameras, although this goal is yet to be attained [13]. Their instrument creates an image of the FOV on a diffuser, placed at the entrance pupil of a kaleidoscope, and based on the working principles of a kaleidoscope reproduces the image of the FOV of the camera many times and uses a spectral bandpass filter array to filter each reproduced image before image acquisition by the camera. Their device acquires multiple spectra for a single viewpoint which provides the benefit of not needing to register the array of images post-acquisition, however it does not maintain the radiometric fidelity of the data. This instrument is therefore limited in performance since it only operates in the visible region of the EM spectrum, is not capable of stereoscopic imaging, and optical aberrations are present in the images which degrade the quality.
Camera approaches for measuring polarization states have to rely upon filters to isolate the direction of vibration in the EM wave. Methods of filtration to measure multiple states include using rotating polarizing filter wheel [14], polarizing prisms [15], and polarizing gratings [16]. Rotating polarizing arrays and polarizing prisms take sequential images, one polarization state at a time, to perform their measurement. In dynamic applications, time differences in the acquisition can require later image registration and signal calibration. These methods can be extended to having multiple detectors dedicated to each polarization state, using various beam splitting mechanisms, to make the image acquisition simultaneous; however, this requires the introduction of additional optics to redirect and split the light, resulting in a more complicated system and potential efficiency issues as well as introducing aberrations in the images. Polarizing gratings have been shown to do simultaneous channelled image polarimetry [16], being able to measuring the Stokes vector of a scene using the interferometric pattern of various polarization states simultaneously. This method uses prior knowledge about the scene's inherent polarization to perform the extraction, thereby requiring pre- and post-calibration of the measuring device in order to extract the desired properties of the measured light.
Light-field imaging is the method of capturing information pertaining to a scene and/or image, where the light intensity going through each point in space at different directions is captured. Light-field imaging has previously been accomplished using multiple cameras positioned at different locations, directions, and focuses, with each camera capturing different light information that is then used to characterize the light-field together. This multi-camera setup is complex, and is difficult to calibrate. Some single-camera light-field imaging systems have been introduced that make use of a microlens array placed behind the main lens of the camera system, directly in front of the detector [17]. In the light-field imaging system proposed by Georgiev [18], an array of lens and/or prisms are placed between the object and/or scene and the main lens of the camera system. It is important to note, however, that none of these light-field imaging systems can capture images at multiple spectral bands that include bands beyond the visible band and at multiple light polarizations, concurrently.
High dynamic range (HDR) imaging is used to overcome the dynamic range limitations of cameras [19]. Modern cameras automatically adjust or allow for adjustment of the exposure time for a specific scene; however, there may exist portions of an image that can be over- or under-exposed as a result. In HDR imaging, this issue is circumvented by sequentially capturing multiple images at varying exposure times [19], using neutral-density (ND) filters to artificially reduce the amount of light that enters the camera while sequentially changing the filter strength [20], or using a series of beam-splitters and camera trees all with different exposure settings [8, 21]. This mode of imaging suffers from similar shortcoming previously mentioned with regards to MS and polarimetric imaging systems.
Each of the above described image capture methods and systems have drawbacks and limitations, as described. Therefore, what is needed are improvements to MS image capture which addresses at least some of these limitations.